Data compression in storage clients

ABSTRACT

Embodiments include method, systems and computer program products for data compression in storage clients. In some embodiments, a storage client for accessing a storage service from a computer program is provided. A compression method is provided in the storage client to reduce a size of data objects. A frequency of compressing data from the computer program or modifying a compression algorithm based on assessing costs and benefits of compressing the data is varied.

BACKGROUND

The present disclosure relates to data storage, and more particularly,to methods, systems and computer program products for data compressionin storage clients.

There are a wide variety of ways of storing data persistently,particularly with cloud-based systems. These include file systems,relational databases (e.g. DB2, MySQL, SQL Server), and NoSQL systems(e.g. Redis, CouchDB/Cloudant, HBase, Hazelcast, MongoDB). It is typicalto have an application program store data persistently using a client.There are a number of problems with storage clients, such as a clientwill typically work for only a single back-end storage system or thatthe performance for accessing the back-end storage systems can besignificant. The problem is often much worse in cloud environments,where the distance to cloud servers can add tens (or even hundreds) ofmilliseconds of latency. In some instances, the persistent storagesystem might become unavailable due to failures or network problems.This can be a problem if the client is communicating remotely with acloud server and does not have good connectivity.

SUMMARY

In accordance with an embodiment, a method for data compression instorage clients is provided. The method may include providing a storageclient for accessing a storage service from a computer program;providing a compression method in the storage client to reduce a size ofdata objects; and varying a frequency of compressing data from thecomputer program or modifying a compression algorithm based on assessingcosts and benefits of compressing the data.

In another embodiment, a computer program product may comprise anon-transitory storage medium readable by a processing circuit andstoring instructions for execution by the processing circuit forperforming a method that may include providing a storage client foraccessing a storage service from a computer program; providing acompression method in the storage client to reduce a size of dataobjects; and varying a frequency of compressing data from the computerprogram or modifying a compression algorithm based on assessing costsand benefits of compressing the data.

In another embodiment, a system may include a processor in communicationwith one or more types of memory. The processor may be configured toprovide a storage client for accessing a storage service from a computerprogram; provide a compression method in the storage client to reduce asize of data objects; and vary a frequency of compressing data from thecomputer program or modify a compression algorithm based on assessingcosts and benefits of compressing the data.

BRIEF DESCRIPTION OF THE DRAWINGS

The forgoing and other features, and advantages of the disclosure areapparent from the following detailed description taken in conjunctionwith the accompanying drawings in which:

FIG. 1 is a block diagram illustrating a computing environment thatincludes multiple storage clients and storage servers in accordance withan exemplary embodiment;

FIG. 2 is a block diagram illustrating an architecture of an enhancedclient in accordance with an exemplary embodiment;

FIG. 3 is a block diagram illustrating a cache interface with multipleimplementations in accordance with an exemplary embodiment;

FIG. 4 is a block diagram illustrating another depiction of a cacheinterfaces with multiple implementations in accordance with an exemplaryembodiment;

FIG. 5 is a block diagram illustrating a remote process cacheimplementation in accordance with an exemplary embodiment;

FIG. 6 is a flow diagram illustrating a method for handling cachedobjects which have expired in accordance with an exemplary embodiment;

FIG. 7 is a diagram illustrating a method for handling poor and/orlimited connectivity in accordance with an exemplary embodiment;

FIG. 8 is a block diagram illustrating a compression interface withmultiple implementations in accordance with an exemplary embodiment; and

FIG. 9 is a block diagram illustrating an encryption interface withmultiple implementations in accordance with an exemplary embodiment; and

FIG. 10 is a block diagram illustrating one example of a processingsystem for practice of the teachings herein.

DETAILED DESCRIPTION

In accordance with exemplary embodiments of the disclosure, methods,systems and computer program products for data compression in storageclients, which offer access to multiple back-end storage systems,improved performance, and higher availability than previous systems. Itis particularly applicable to the cloud where there are multiple storageservices available and latency for accessing a cloud storage service canbe high.

The systems and methods described herein may provide enhanced storagecapabilities, a broad selection of storage options, optimize latency foraccessing cloud storage systems (e.g., move significant data handlingcapabilities into client), may avoid overhead of remote storage, and mayavoid sending confidential data.

In some embodiments, an application may use multiple cloud storagesystems or change from using one cloud storage system to another. Acloud storage manager may be provided as a layer above the cloud storagesystem, which allows an application to easily use multiple cloud storagesystems and provides additional services not provided by cloud storagesystems. The cloud storage manager may provide a storage interface forapplications to use. The storage interface may be built for each cloudstorage system of interest. In some embodiments, applications may accesscloud storage through the storage interface. Substituting differentcloud storage systems may not require changes to an application. Optionsfor key-value stores, relational databases, and file systems may beprovided by the cloud storage manager.

The methods and systems described herein are directed to the design andimplementation of storage clients, which offer access to multipleback-end storage systems, improved performance, and higher availabilityof storage capabilities. It is particularly applicable to the cloudwhere there are multiple storage services available and latency foraccessing a cloud storage service can be high.

In some embodiments, the storage client may handle multiple back-endsystems. The storage client may define a key-value interface. Anyback-end storage system, which implements the key-value interface, mayuse the storage client.

If the server supports delta encoding, then the server may make thechoice as to whether to decode a delta and store the full object or tojust store the delta. In many cases, the server may not have the abilityto decode a delta. In this case, the client may instruct the server tosimply store a delta from the previous version. After a certain numberof deltas, the client may send a full object (not just the delta) to theserver. That way, the server does not have to keep accumulating deltas.Note that the client may perform all delta encoding and decoding (ifnecessary). The server does not have to understand how to perform deltaencoding or decoding.

The systems and methods described herein may provide encryption. Usersmight desire all data stored persistently to be encrypted. Therefore,the storage client may provide data encryption and decryptioncapabilities.

Some embodiments of the disclosure may be directed to support users whohave poor connectivity. The caches described herein may provide a methodfor users to continue to run an application when connectivity is poor.When connectivity is restored, a remote storage service can be updatedin batches.

FIG. 1 is a block diagram illustrating a computing environment 100 thatincludes multiple storage clients and storage servers in accordance withan exemplary embodiment. In some embodiments, storage systems (which maybe offered over the cloud) such as Cloudant, Object Storage (whichimplements the OpenStack Swift API), and Cassandra typically haveclients (e.g., Cloudant client 115, Object Storage client 125, Cassandraclient 135), which application programs use to communicate with theactual storage servers (e.g., Cloudant server(s) 110, Object Storageserver(s) 120, Cassandra server(s) 130). Although this disclosure isdiscussed in the context of cloud storage systems, the systems andmethods described herein may be applicable to other types of storagesystems. In some cases, the clients (e.g., Cloudant client 115, ObjectStorage client 125, Cassandra client 135) can be language specific (e.g.written for a specific programming language, such as Java, Python,JavaScript). For example, a Java client might be designed with an APIallowing Java programs to use the API using Java method calls. Otherstorage clients have other types of API's. For example, a Rest API wouldallow applications to access a storage system using HTTP. The systemsand methods described herein may be compatible with a wide variety oftypes of client (and server) APIs for accessing storage systems,including but not limited to method and/or function calls fromconventional programming languages, protocols (e.g. HTTP, XML, JSON,SOAP, many others), and several other established methods for specifyinginterfaces.

Although the disclosure discusses Cloudant, Object Storage, andCassandra, these services are merely exemplary and other systems andmethods described herein may be applied to different cloud or remotesystems or services.

FIG. 2 is a block diagram illustrating an architecture 200 of anenhanced storage client in accordance with an exemplary embodiment. Theenhanced storage client may handle multiple back-end systems. Theenhanced storage client may include the enhanced client module 210 andthe cloud service subclients (e.g., Cloudant subclient 215, ObjectStorage subclient 220, Cassandra subclient 225). This enhanced storageclient allows application programs to communicate with multipledifferent back-end storage systems (e.g., Cloudant server(s) 110, ObjectStorage server(s) 120, Cassandra server(s) 130).

In some embodiments, a key-value interface may be implemented for theenhanced storage client, which may be standardized across all back-endstorage systems. Any back-end storage system may use the key-valueinterface by implementing a subclient (e.g., Cloudant subclient 215)that implements the key-value interface over a back-end storage system(e.g., Cloudant server(s) 110). In this case, an application program canuse the back-end storage system by communicating with the enhancedclient. It should be noted that the subclient (e.g., Cloudant subclient215) may implement other methods for communicating with the back-endstorage system beyond just the key-value interface. The application hasthe option of using the back-end-specific methods in the subclient forcommunicating with the back-end storage system, in addition to theenhanced client key-value interface, which is standard across allback-end storage systems. That way, the application program still hasthe full generality of the features for the back-end storage system. Thekey-value interface does not limit the usage of the back-end storagesystem by an application program, since the application program canbypass the key-value interface and use the back-end storagesystem-specific API calls from the subclient. Other implementations(besides key-value interfaces) are also possible for the enhancedclients.

FIGS. 3-4 are discussed collectively. Caching be used to improveperformance and may be useful in cloud-based storage systems in whichthe client (e.g., Cloudant client 115) is remote from the storage server(e.g., Cloudant server(s) 110). In such embodiments, the physicaldistance between the client and server may add to the latency forstorage operations. In some embodiments, the caches may be integrateddirectly with the client (e.g., Cloudant client 115, Object Storageclient 125, Cassandra Client 135), which may enhance functionality andperformance of the clients. Additionally, the integration of the cacheswith the clients may be a feature for application programmers. Ifapplication programmers have to implement their own caching solutionsoutside of the client, it may require considerably more work, and theperformance of such caching solutions may not be as good.

FIG. 3 is a block diagram illustrating an environment 300 with a cacheinterface 310 with multiple implementations in accordance with anexemplary embodiment. Multiple caches may be used within the enhancedstorage clients. In some embodiments, to utilize a particular cache, thecache interface 310 may be implemented on top of the particular cache.The modular cache design may include a cache interface 310, a sameprocess implementation 315 as the client (e.g., an in-process cache,which may store data in the same process as the application program), aremote process(es) 320, which may be an open source cache such as Redis415 and memcached 425, and other implementations 325 (e.g., an opensource cache such as Ehcache or Guava caches). The in-process cache 420may store data in the same process as the application program.

FIG. 4 is a block diagram illustrating another environment 400 of acache interfaces 410 with multiple implementations in accordance with anexemplary embodiment. In some embodiments, the cache design may bemodular. Multiple caches may be used within our enhanced clients. Inorder to use a particular cache, the cache interface 410 should beimplemented on top of a cache (e.g., as illustrated in FIGS. 3-4). Thein-process cache 420 may store data in the same process as theapplication program.

FIG. 5 is a block diagram illustrating a remote process cacheimplementation 500 in accordance with an exemplary embodiment. In someembodiments, two types of caches may be utilized: in-process and remoteprocess. In-process caches may operate in the same process as theapplication process. They have the advantage of being fast. Data (e.g.,cached objects) does not need to be serialized in order to be cached.The cache is not shared with other clients or applications.

Remote process caches 520 (e.g. Redis, memcached) execute in differentprocesses from the application program. They have the advantage thatthey can be shared by multiple clients (e.g., Client1 510, Client2 515)and applications. Furthermore, they can scale to many processes (whichcan execute on the same or distinct computing nodes). On the negativeside, there is some overhead for the interprocess communication that isrequired for applications/clients to communicate with the cache(s) 520.In addition, cached data may need to be serialized, which introducesadditional overhead.

When the cache 520 becomes full, a method may be needed to determinewhich object to remove from the cache 520 to make room for otherobjects. This process is known as cache replacement. One of the mostwidely used cache replacement algorithms is to replace the object whichwas accessed most distantly in the past (least recently used, or LRU).Other cache replacement algorithms (e.g. greedy-dual size) are alsopossible. Different cache replacement algorithms are also compatiblewith the methods and systems described herein.

FIG. 6 is a flow diagram illustrating a method 600 for handling cachedobjects, which have expired in accordance with an exemplary embodiment.In some embodiments, cached objects may have expiration times associatedwith them. Once the expiration time for a cached object has passed, theobject is no longer valid. Cached objects may be deleted from the cacheafter they have expired. Alternatively, they can be kept in the cacheafter their expiration times, permitting cached objects that haveexpired but are still current to remain in the cache.

For example, at block 605, an object (o1) with an expiration time of7:00 AM may be cached at 6:00 AM. At block 610, at 7:00 AM, o1 mayremain cached. At block 615, at 7:04 AM, o1 may be requested. When o1 isrequested, the server is contacted to see if the version of o1 in thecache is still current (e.g., a get-if-modified-since request may betransmitted to the server). If the server indicates that o1 is stillcurrent, the method may proceed to block 620, where the expiration timeassociated with o1 is updated using a new expiration time provided bythe server. This may save network bandwidth (depending on the size ofo1) since o1 does not need to be unnecessarily fetched from the server.If the cached version of o1 is determined to be obsolete at block 615,then the method may proceed to block 625. At block 625, the server maysend an updated version of o1 to the client, and the cache may beupdated using the updated version of o1 received from the server.

FIG. 7 is a diagram illustrating a method 700 for handling poor and/orlimited connectivity in accordance with an exemplary embodiment. In someembodiments, the cache 710 integrated with the enhanced storage clientmay be used to mitigate connectivity problems between the client 705 andserver 715. During periods when the server 715 is unresponsive (e.g.,not responding within a predetermined length of time) and/or the cost tocommunicate with the server is high (e.g., resources exceed anpredetermined threshold), an application (which may be implemented byone or more computer programs) can operate by using the cache 710 forstorage instead of the server 715. At data exchanges 720 and 725, theclient 705 and server 715 may communicate to transmit batch updates orinitiate synchronization when the connectivity is deemed to beresponsive (e.g., server 710 responds to a client request within apredetermined period of time).

FIG. 7 depicts a situation in which an application using the client 705relies upon the cache 710 when connectivity between the client 705 andthe server 715 is poor. If the server(s) 715 are not responding or isresponding too slowly, the application may use the cache 710 for storageinstead of the server 715. When server 715 response times improve, theapplication may start using the storage server 715 again. At this point,several messages might have to be exchanged between the client 705 andthe server 715 to make the contents of the cache 710 and the cloudstorage server(s) 715 consistent. For example, if the application hasupdated o1 to version v2 at 7:35 in the cache 710 and the cloud storageserver 715 has a previous version of o1 from 7:30, then version v2 of o1is stored at the cloud storage server(s) 715. If the cache 710 isstoring o2 version v3 and the storage server 715 has a newer version v4of o2, then version v4 of o2 is stored in the cache 710.

The client 705 may make adaptive decisions of how frequently to use thecache 710 based on the responsiveness of the server 715. The client 705may monitor the time it takes for server(s) 715 to respond to clientrequests. When the server 715 is slow to respond, the client 705 canincrease the frequency it uses for caching data. When the server 715 isresponding relatively quickly without significant delays, the client 705can decrease the frequency it uses for caching. For example, supposethat the average time for getting a response from the server 715increases by 70%. This might result in the client 705 increasing thepercentage of requests that it directs to the cache 710. The client 705might choose to store data in the cache 710 more frequently. It mightalso choose to retrieve data from the cache 710 more frequently withoutchecking with the server 715 to determine if the cached data is the mostcurrent version.

Suppose the average time for getting a response from the server 715decreases by 50%. The client 705 might choose to use the cache 710 lessfrequently. For example, it might store data more frequently at theserver 715 instead of caching it. It might also choose to morefrequently check with the server 715 to determine if a cached object iscurrent.

FIG. 8 is a block diagram illustrating a compression interface 805 withmultiple implementations in accordance with an exemplary embodiment. Insome embodiments, the enhanced storage client may be used to reduce theoverhead of large data objects. This may be handled by both datacompression and delta encoding. The enhanced storage client may have theability to compress objects prior to storage and to decompress them uponretrieval.

In some embodiments, the compression design may be modular. In someembodiments, the compression interface 805 may be defined. Multiplecompression algorithms may be used within our enhanced clients. In orderto use a particular compression algorithm, the compression interface 805may be implemented on top of the compression algorithm.

In some embodiments, compression techniques described herein may includeadaptive compression in which the amount and degree of compression canbe varied based on run-time conditions. In some embodiments, it may bedesirable to perform compression when cache space is low (e.g., below apredetermined threshold, where the threshold may be modified by a user),since a compressed object takes up less cache space. Similarly, it maybe desirable to perform compression when space in the storage service islow, since a compressed object takes up less storage service space.Sometimes, there is a cost to storing data with the storage service. Ifthis cost goes up, it becomes more desirable to perform compressionbefore storing a data object with the storage service.

In some embodiments, the bandwidth between the client and the server canaffect performance. When that bandwidth is low (e.g., below apredetermined threshold), it becomes more desirable to compress dataobjects before sending them from the client to the server. Compressionmay take up CPU cycles. Therefore, when the client CPU is heavilyutilized, it may be less desirable to perform compression. When theclient CPU is lightly utilized, it becomes more desirable to performcompression.

Not all data objects compress equally well. The client can predict fromthe type of a data object whether it is a good candidate for compressionbased on empirical evidence it has on how well similar data objects havecompressed in the past. If the client determines that little space islikely to be saved by compressing a data object (o1), then it may not bedesirable to compress o1, as doing so would incur some CPU overhead. Ifthe client determines that considerable space can be saved bycompressing o1, then it may be desirable to compress o1. Examples ofcompression algorithms that may be used by the compression interface 805may include, but are not limited to Snappy 810, Iz4 815, and/or gzip820.

In some embodiments, the client may control the amount of compression byvarying the frequency with which it will compress a data object. If theclient determines that compression is desirable, it may compress dataobjects frequently (e.g., a set number of times during a given timeperiod). If the client determines that compression is not desirable, itcan compress data objects less frequently (e.g., fewer times in a giventime period).

In some embodiments, the enhanced storage clients may allow differenttypes of compression algorithms to be used. Some compression algorithmsare efficient at compressing data, while others are not as efficient buthave the advantage of using fewer CPU cycles. The data compression ratiois the uncompressed data size of an object divided by the compresseddata size. The data compression ratio is dependent on both the dataobject and the compression algorithm. In general, an algorithm with ahigher compression ratio will result in more compression at the cost ofhigher CPU overhead. If a data compression algorithm consumes more CPUcycles without improvement in compression ratio, it is probably not agood algorithm to use.

The enhanced storage client may have the capability to increase theamount of compression, via some combination of increasing the frequencyof data compression and/or using data compression algorithm(s) with ahigher compression ratio(s) in response to one or more of the following:

-   -   1. The amount of free cache space available to the computer        program falls below a threshold.    -   2. The amount of free space in the storage service available to        the computer program falls below a threshold.    -   3. Available bandwidth between the computer program and the        storage service falls below a threshold.    -   4. A cost for storing data on the storage service increases.    -   5. The type of the data object currently being stored has a        higher compression ratio.    -   6. The CPU utilization of the client decreases.

The enhanced storage clients may have the capability to decrease theamount of compression, via some combination of decreasing the frequencyof data compression and/or using data compression algorithm(s) withlower CPU overhead (which generally means a lower compression ratio) inresponse to one or more of the following:

-   -   1. The amount of free cache space available to the computer        program rises above a threshold.    -   2. The amount of free space in the storage service available to        the computer program rises above a threshold.    -   3. Available bandwidth between the computer program and the        storage service rises above a threshold.    -   4. A cost for storing data on the storage service decreases.    -   5. The type of the data object currently being stored has a        lower compression ratio.    -   6. The CPU utilization of the client increases.

In some embodiments, the overhead may be reduced by delta encoding.Delta encoding is useful when a client is sending updated objects to theserver. Instead of sending the full object each time, the client cansend only a delta (e.g., the difference between the current version andthe last stored version on the server). In many cases, deltas are only asmall fraction of the size of the complete object.

If the server supports delta encoding, then the server may make thechoice as to whether to decode a delta and store the full object or tojust store the delta. In many cases, the server will not have theability to decode a delta. The client can instruct the server to simplystore a delta from a previous version. After a certain number of deltas,the client can send a full object (not just the delta) to the server.That way, the server does not have to keep accumulating deltas. Notethat the client can perform all delta encoding and decoding (ifnecessary). The server does not have to understand how to perform deltaencoding or decoding.

Delta encoding enables a client storing multiple updates to an object(o1) to send the deltas (e.g., changes) resulting in the new objects d₁,d₂, . . . , d_(n) instead of the entire copies of the updated objects.Accordingly, less information needs to be sent from the client to theserver. At some point, the client might send a full updated objectinstead of a delta. There are multiple ways in which a client might makea decision to send a full version of an object instead of a delta:

-   -   1. The client might wait until the number of previous deltas it        has sent since sending the last full version of the object has        exceeded a threshold.    -   2. The client might wait until the total number of bytes        contained in deltas exceeds a threshold.    -   3. The client might make a determination of the cost to        construct an updated object by applying deltas to the previous        version of the object stored at the server. Once this cost        exceeds a threshold, the client then sends a full version of the        object instead of a delta.    -   4. The decision can also be made in a large number of other ways        within the spirit and scope of the invention.

When a client sends a delta d_(i) for o1, the server may retain theprevious version of o1 and all other deltas needed to construct theupdated version of o1 from the previous version. In order to reconstructan updated version of o1 from an earlier version, deltas are used toconstruct the updated version of o1. The updated version can beconstructed by the server, the client, or by another party. The factthat the server does not have to apply deltas to reconstruct the objectmeans that enhanced storage clients can use delta encoding without theserver having special support for delta encoding. The server can beunaware that delta encoding is actually being implemented.

In some cases, a client may have to determine the value of o1 when itdoes not have a copy of o1 stored locally. Instead, o1 is represented bya previous version and multiple deltas on the server. If the server doesnot have the capability to decode deltas, the previous version of o1 andthe subsequent deltas can be retrieved by the client from the server.The client then determines an updated version of o1 by applying thedeltas to the previous version of o1.

When the client successfully stores a full updated version of o1 insteadof a delta on the server, the previous version of o1 stored on theserver, as well as previous deltas applicable to this previous versionof o1, can be deleted from the server. This saves space.

FIG. 9 is a block diagram illustrating an environment 900 with anencryption interface with multiple implementations in accordance with anexemplary embodiment. In some embodiments, users may want all datastored persistently to be encrypted, which may be provided by theenhanced storage client. In some embodiments, the encryption design maybe modular. An encryption interface may be defined 905. Multipleencryption algorithms can be used within the enhanced clients. In orderto use a particular encryption algorithm, the encryption interface maybe implemented on top of the particular encryption algorithm. In someembodiments, users can encrypt data using encryption algorithms, such asAES 128 bits 910, AES 256 bits 915, or Blowfish 448 bits 920. Using theenhanced storage client, users may encrypt data before it is ever storedin a server, or before the data is cached.

Referring to FIG. 10, there is shown an embodiment of a processingsystem 1000 for implementing the teachings herein. In this embodiment,the system 1000 has one or more central processing units (processors)1001 a, 1001 b, 1001 c, etc. (collectively or generically referred to asprocessor(s) 1001). In one embodiment, each processor 1001 may include areduced instruction set computer (RISC) microprocessor. Processors 1001are coupled to system memory 1014 and various other components via asystem bus 1013. Read only memory (ROM) 1002 is coupled to the systembus 1013 and may include a basic input/output system (BIOS), whichcontrols certain basic functions of system 1000.

FIG. 10 further depicts an input/output (I/O) adapter 1007 and a networkadapter 1006 coupled to the system bus 1013. I/O adapter 1007 may be asmall computer system interface (SCSI) adapter that communicates with ahard disk 1003 and/or tape storage drive 1005 or any other similarcomponent. I/O adapter 1007, hard disk 1003, and tape storage device1005 are collectively referred to herein as mass storage 1004. Operatingsystem 1020 for execution on the processing system 1000 may be stored inmass storage 1004. A network adapter 1006 interconnects bus 1013 with anoutside network 1016 enabling data processing system 1000 to communicatewith other such systems. A screen (e.g., a display monitor) 1015 isconnected to system bus 1013 by display adaptor 1012, which may includea graphics adapter to improve the performance of graphics intensiveapplications and a video controller. In one embodiment, adapters 1007,1006, and 1012 may be connected to one or more I/O busses that areconnected to system bus 1013 via an intermediate bus bridge (not shown).Suitable I/O buses for connecting peripheral devices such as hard diskcontrollers, network adapters, and graphics adapters typically includecommon protocols, such as the Peripheral Component Interconnect (PCI).Additional input/output devices are shown as connected to system bus1013 via user interface adapter 1008 and display adapter 1012. Akeyboard 1009, mouse 1010, and speaker 1011 all interconnected to bus1013 via user interface adapter 1008, which may include, for example, aSuper I/O chip integrating multiple device adapters into a singleintegrated circuit.

In exemplary embodiments, the processing system 1000 includes agraphics-processing unit 1030. Graphics processing unit 1030 is aspecialized electronic circuit designed to manipulate and alter memoryto accelerate the creation of images in a frame buffer intended foroutput to a display. In general, graphics-processing unit 1030 is veryefficient at manipulating computer graphics and image processing, andhas a highly parallel structure that makes it more effective thangeneral-purpose CPUs for algorithms where processing of large blocks ofdata is done in parallel.

Thus, as configured in FIG. 10, the system 1000 includes processingcapability in the form of processors 1001, storage capability includingsystem memory 1014 and mass storage 1004, input means such as keyboard1009 and mouse 1010, and output capability including speaker 1011 anddisplay 1015. In one embodiment, a portion of system memory 1014 andmass storage 1004 collectively store an operating system such as theAIX® operating system from IBM Corporation to coordinate the functionsof the various components shown in FIG. 10.

The present disclosure may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent disclosure.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present disclosure may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A computer-implemented method comprising:providing a storage client for accessing a storage service from acomputer program, wherein the storage client is implemented withsoftware and the storage service is running on at least one computer;the storage client storing data using the storage service; providing acompression method, implemented with software, in the storage client toreduce a size of data objects; and varying a frequency of compressing,using the compression method, at least some of the data from thecomputer program based on assessing, with the storage client software,costs and benefits of compressing the data; and wherein varying thefrequency of compressing data further comprises at least one of: a)increasing the frequency of compressing data or using a compressionalgorithm with a higher data compression ratio, when the storage clientdetermines that an available bandwidth between the storage client andthe storage service falls below a threshold; or b) monitoring a CPUusage on at least one computing node performing compression; andincreasing the frequency of compressing data or using a compressionalgorithm with a higher data compression ratio, when the storage clientdetermines that the monitored CPU usage decreases.
 2. Thecomputer-implemented method of claim 1, wherein varying the frequency ofcompressing data further comprises: increasing the frequency ofcompressing data or using a compression algorithm with a higher datacompression ratio, when an amount of free cache space available to thecomputer program falls below a threshold.
 3. The computer-implementedmethod of claim 1, wherein varying the frequency of compressing datafurther comprises: increasing the frequency of compressing data or usinga using a compression algorithm with a higher data compression ratio,when the an amount of free space in the storage service available to thecomputer program falls below a threshold.
 4. The computer-implementedmethod of claim 1, wherein varying the frequency of compressing datafurther comprises: increasing the frequency of compressing data or usinga compression algorithm with a higher data compression ratio, when acost for storing data on the storage service increases.
 5. Thecomputer-implemented method of claim 1, wherein varying the frequency ofcompressing data further comprises: determining data compression ratiosfor different datatypes; and increasing the frequency of compressingdata or using a compression algorithm with a higher data compressionratio, for datatypes with higher data compression ratios.
 6. Thecomputer-implemented method of claim 1, further comprising: providing anencryption method in the storage client to preserve data privacy.
 7. Thecomputer-implemented method of claim 1, further comprising: providing acache within the storage client for reducing a number of accesses to thestorage service.
 8. A computer program product comprising anon-transitory storage medium readable by a processing circuit andstoring instructions for execution by the processing circuit forperforming a method comprising: providing a storage client for accessinga storage service from a computer program, wherein the storage client isimplemented with software and the storage service is running on at leastone computer; the storage client storing data using the storage service;providing a compression method, implemented with software, in thestorage client to reduce a size of data objects; and varying a frequencyof compressing data from the computer program or modifying a compressionalgorithm, based on assessing costs and benefits of compressing thedata; and wherein varying the frequency of compressing data furthercomprises at least one of: a) increasing the frequency of compressingdata or using a compression algorithm with a higher data compressionratio, when the storage client determines that an available bandwidthbetween the storage client and the storage service falls below athreshold; or b) monitoring a CPU usage on at least one computing nodeperforming compression; and increasing the frequency of compressing dataor using a compression algorithm with a higher data compression ratio,when the storage client determines that the monitored CPU usagedecreases.
 9. The computer program product of claim 8, wherein varyingthe frequency of compressing data further comprises: increasing thefrequency of compressing data or using a compression algorithm with ahigher data compression ratio, when an amount of free space in thestorage service available to the computer program falls below athreshold.
 10. The computer program product of claim 8, wherein varyingthe frequency of compressing data further comprises: increasing thefrequency of compressing data or using a compression algorithm with ahigher data compression ratio, when an available bandwidth between thecomputer program and the storage service falls below a threshold. 11.The computer program product of claim 8, wherein varying the frequencyof compressing data further comprises: increasing the frequency ofcompressing data or using a compression algorithm with a higher datacompression ratio, when a cost for storing data on the storage serviceincreases.
 12. The computer program product of claim 8, wherein varyingthe frequency of compressing data further comprises: determining datacompression ratios for different datatypes; and increasing the frequencyof compressing data or using a compression algorithm with a higher datacompression ratio, for datatypes with higher data compression ratios.13. The computer program product of claim 8, the method furthercomprising: providing an encryption method in the storage client topreserve data privacy.
 14. A system, comprising: a processor incommunication with one or more types of memory, the processor configuredto: provide a storage client for accessing a storage service from acomputer program, wherein the storage client is implemented withsoftware and the storage service is running on at least one computer;the storage client stores data using the storage service; provide acompression method, implemented with software, in the storage client toreduce a size of data objects; and vary a frequency of compressing,using the compression method, at least some of the data from thecomputer program based on assessing, with the storage client software,costs and benefits of compressing the data; and wherein varying thefrequency of compressing data further comprises at least one of: a)increasing the frequency of compressing data or using a compressionalgorithm with a higher data compression ratio, when the storage clientdetermines that an available bandwidth between the storage client andthe storage service falls below a threshold; or b) monitoring a CPUusage on at least one computing node performing compression; andincreasing the frequency of compressing data or using a compressionalgorithm with a higher data compression ratio, when the storage clientdetermines that the monitored CPU usage decreases.
 15. The system ofclaim 14, wherein, to vary the frequency of compressing data, theprocessor is further configured to: increase the frequency ofcompressing data or use a compression algorithm with a higher datacompression ratio, when an amount of free cache space available to thecomputer program falls below a threshold.
 16. The system of claim 14,wherein, to vary the frequency of compressing data, the processor isfurther configured to: increase the frequency of compressing data or usea use a compression algorithm with a higher data compression ratio; whenthe an amount of free space in the storage service available to thecomputer program falls below a threshold.